Skip to content

TEST PR ! Wayland native backend - #1018

Closed
alexei-zebra wants to merge 17 commits into
MihailRis:mainfrom
alexei-zebra:wayland-native-backend
Closed

alexei-zebra wants to merge 17 commits into
MihailRis:mainfrom
alexei-zebra:wayland-native-backend

Conversation

@alexei-zebra

Copy link
Copy Markdown
Contributor

No description provided.

Wayland support was missing a proper windowing layer: the GLFW backend
leaves the known platform gaps (cursor themes, window icon, window
position, fullscreen output selection).

The new backend talks to the compositor directly:
- xdg-shell, EGL, xkbcommon (keyboard, repeat, layout) instead of GLFW;
- cursors through wp_cursor_shape_manager_v1, so no cursor theme lookup
  is required and no "standard cursor shape unavailable" errors appear;
- window icon and title are drawn by the engine itself, since Wayland
  has no window icon protocol and GNOME provides no server-side
  decorations;
- fullscreen requests the current output (set_fullscreen with NULL), so
  the window no longer jumps to another monitor;
- interactive resize presents a cached frame scaled to the new size
  while the pointer is down and refreshes the real frame when the drag
  pauses, keeping the window glued to the cursor.

libdecor is not used: libdecor_frame_commit() redraws the whole
decoration and acks the configure in the same call, and mutter sends
200-330 configures per second during an interactive resize, so the
decoration was redrawn hundreds of times per second (measured: 2 s of
work per second, loop stalls up to 6 s). The decorations are drawn by
the engine instead, which measured 2.3 ms average configure-to-present
latency during a real drag.

GLFW stays the backend for X11 and Windows; VOXEL_WINDOW_BACKEND=x11
forces it on Wayland sessions.
WaylandWindow.cpp held the protocol bindings, the shared compositor
state, the input implementation, the window itself, the free event
listeners and the initialization code in one 2400-line file.

Moved out of it:
- WaylandCommon.hpp/.cpp: shared compositor state, the display event
  loop, keysym mapping and the window logger;
- WaylandInput.hpp: the input implementation (keyboard, clipboard,
  cursors, key repeat).

No functional change.
WaylandWindow.cpp kept the window class with all methods defined in
place, the decorations, the free event listeners and the initialization
in one file. Now:

- WaylandWindow.hpp: the class declaration;
- WaylandWindow.cpp: surface, EGL, resize and frame pacing;
- WaylandBar.cpp: the window decorations (bar, font, buttons, dragging,
  edge resizing);
- WaylandListeners.cpp/.hpp: the free Wayland event handlers;
- WaylandInit.cpp: wayland_window_initialize.

No functional change.
The bar used to be drawn by the Wayland backend with its own shader,
bitmap font and textures, straight into the engine's GL frame. That
leaked GL state (the batch caches bound textures, so the engine kept
sampling the backend texture and all text turned into solid white
rectangles) and it duplicated fonts and styling.

Now the backend only owns the behaviour: hit zones, cursor shapes,
click interception and the compositor requests (move, resize,
minimize, maximize/restore, close). The bar itself is drawn by the
engine in WindowBar.cpp through the regular Batch2D/Font path, so it
looks like the rest of the UI.

Window gained getTitle(), getDecorationHeight() and
getDecorationHoveredButton(); GLFW keeps the defaults, so X11 and
Windows are unaffected. The backend's decoration drawing (bitmap font,
shader, quad batches) is removed: WaylandBar.cpp went from 630 to 140
lines.

Fixes found while debugging:
- GlStateGuard was defined but never used after the file split, so the
  stretched frame left its texture bound and broke text rendering after
  a window mode change; it is now applied around the cache capture and
  the stretched frame;
- takeScreenshot() now reads the front buffer and restores the read
  buffer afterwards.
The backend sources live in window/wayland and were picked up by the
global source glob, so a build with VOXELENGINE_WAYLAND=OFF (and any
non-Linux build, where the option is skipped entirely) tried to compile
them and failed on the generated protocol headers.

They are now collected separately and added to the target only when the
backend is enabled.

Also restores the CRLF line endings of Engine.cpp, GUI.hpp and
input.cpp, which were rewritten with LF and made their diffs look like
whole-file rewrites.
Keeps the interface non-breaking for other window backends: the
decoration height and hovered button already have defaults, and now so
does the title.
The backend, decoration and resize-logging switches were only used
while working on this, and undocumented ones should not stay:

- VOXEL_WINDOW_BACKEND: the backend is still chosen by WAYLAND_DISPLAY,
  and GLFW is used when it is not set;
- VOXEL_DECORATIONS: own decorations are always enabled when the
  compositor provides none;
- VOXEL_LOG_RESIZE: the resize statistics are removed together with the
  counters they needed.

WaylandDecorations.cpp replaces WaylandBar.cpp: drawing moved to the
engine, the file only holds the decoration behaviour now.
getClipboardText() used to open a pipe, ask the selection owner for
the text and poll it with a 500 ms deadline, so every paste blocked the
UI thread (TextBox calls it directly) and a slow owner meant a visible
hang.

The selection is now fetched in the background: when the selection
changes, the backend requests the text into a non-blocking pipe and
getClipboardText() only returns the buffer, which the event loop fills
in as the data arrives. Pasting no longer blocks, and a source that
never answers simply leaves the previous buffer in place.

Also clears the advertised MIME list when a new selection offer
arrives, so a request is never made with MIME types of a previous
owner.
The workflow only ran for pushes to main/release and for pull requests,
so a topic branch could not be built without opening a pull request
first.
Both backends asked for an alpha channel (GLFW defaults to 8 alpha
bits, the Wayland EGL config requested 8), so everything the engine
draws with alpha - the UI panels, the water post effect - lowered the
window's own alpha and the compositor blended the desktop through it.
Visible on wine/Proton, where the whole window is composited.

Both now ask for an opaque framebuffer: GLFW_ALPHA_BITS 0, and
EGL_ALPHA_SIZE 0 with a fallback to an alpha config if the driver
provides none. Checked the result through the back buffer attachment:
alpha size is 0 on Wayland and on X11.
The native Wayland backend had no window controls: no title, no close
button, no way to move or resize the window, and the fullscreen toggle
could throw the window to another monitor. GNOME does not implement
zxdg_decoration_manager_v1, so there is nobody to draw a frame for us:
the backend now falls back to its own decorations when the compositor
offers no server side ones, and the engine draws the visible part.

The bar itself is drawn by the engine (WindowBar.cpp) through the usual
batch and font, so it uses the same assets as the rest of the UI and the
backend only owns the behaviour: hit zones, cursor shapes, click
interception and the xdg-shell requests. Its geometry follows the
desktop's button layout, read and subscribed through
xdg-desktop-portal over sd-bus (libsystemd is an optional dependency,
gsettings is only a startup fallback).

Controls behave like they do elsewhere:
- pressing anywhere on the bar starts a move once the pointer leaves a
  5 px radius, so dragging works from the buttons as well;
- a button acts on release, only if the pointer stayed on it;
- a double click on the bar toggles maximize;
- the top edge and the top corners resize, the top 8 px of the sides,
  the sides and the bottom are 8 px wide resize strips;
- right and middle clicks and the scroll wheel over the bar never reach
  the game.

The in-game user interface is offset below the bar while menus keep the
whole window, so the translucent bar shows the menu background through
itself and never covers the HUD. The HUD lays its elements out against
the content height and the cursor is translated once, which fixes items
following the mouse one row above the pointer in game.

The window size is now written to the settings when the window is
resized in windowed mode and applied on the first configure, so the
window opens with the size it was left with, and leaving fullscreen or
borderless mode restores it instead of overwriting the settings with the
fullscreen size (an unconditional unmaximize made the compositor restore
the maximized geometry).

Measured: menu layout unchanged (706 bright pixels in the same place),
gui viewport 1123x751 in menus against 1123x721 in game, 1123x751 ->
2560x1080 -> 1123x751 round trip keeps the settings, a saved 1000x640
opens as 1000x640, no errors in the log.
The engine-drawn title bar grew to 48 px with 24x24 window controls,
12 px away from the edge and from each other, all sharing one hover
highlight. The controls use textures now (gui/cross, gui/minimize_w,
gui/maximize_w and gui/restore_w when maximized) with the vector icons
kept as a fallback, and the title is centred in the bar, clamped to the
free space between the button groups.

The bar now sits inside invisible margins that carry a drop shadow, so
the window is resizable from outside the visible frame: the surface is
content plus twice SHADOW_MARGIN and xdg_surface.set_window_geometry
tells the compositor which part is the window. The shadow follows
Adwaita's window shadow: a Gaussian blur of 7 px (sigma 3.5) expanded by
1 px, rgba(0,0,0,0.5) focused and 0.3 in the backdrop, plus a 1 px
hairline, symmetric on all four sides - a 16 px wide frame would show a
fatter tail than the reference. Margins exist only while the window is
decorated by us and not maximized or fullscreen.

The engine composes the frame itself: it renders exactly as before, then
the backend copies the frame into a texture, clears the surface, draws
the shadow and puts the content in place with alpha forced to 1, which
keeps the frame opaque without touching a single render pass. Only
frames the engine actually drew are cached, and the EGL window is
resized before its surface size is asked for, so no frame is composed
from a buffer the compositor just replaced - that was the one frame
blink after a resize.

Resize cursors match GNOME: directional shapes for edges and corners
(w/e/n/s/ne/nw/se/sw) instead of the double-headed diagonals, with the
extra CursorShape values appended so the existing GLFW indices stay.

Measured: window 1123x751 opens as saved, fullscreen round trip returns
the same size with the settings untouched, the shadow alpha profile is
0.20/0.10/0.03 at 1/4/7 px from the edge against Adwaita's 0.21/0.095/
0.055, all four sides equal, menu layout baseline unchanged (706),
no errors in the log.
@alexei-zebra
alexei-zebra force-pushed the wayland-native-backend branch from 75068da to f43b993 Compare October 9, 2026 19:28
A review of the branch turned up a number of defects, gaps and leftovers;
this commit addresses all of them.

Defects:
- the shadow uniform array held seven entries while eight locations were
  written, so the last one landed outside the object;
- CursorShape::LAST stayed in the middle of the enum after the new corner
  shapes were appended, which made the GLFW cursor table read out of
  bounds; the extra shapes are now mapped onto the ones GLFW knows, and
  the table is only filled for those;
- GlStateGuard asked glIsEnabled for an attribute pname, so it always got
  GL_INVALID_ENUM and never restored the vertex attribute arrays;
- the early returns in wayland_window_initialize left the global window
  and input pointers dangling when falling back to GLFW;
- the icon textures were listed nowhere, so none of them ever loaded;
- a shader failure disabled the shadow after the surface had already been
  sized for it, and VOXEL_DECORATIONS outlived the switch cleanup.

Behavior:
- the keyboard now releases every pressed key when it loses focus, and a
  repeat rate of zero means no repeat;
- the fourth UTF-8 lead byte is decoded with the right mask, control
  characters no longer reach text fields, and a vanished clipboard offer
  clears the cached text;
- compositor side decorations disable ours again instead of leaving two
  titles, the window destructor releases the rest of the state, and the
  content inset is computed before the GUI acts so the HUD and the
  container never disagree for a frame; the HUD clamps it;
- the window controls fall back to minimize, maximize and close, and the
  icons are drawn from textures only.

Cleanups: one monotonic timer instead of three, one vertex shader shared
by both renderers with a link check, one button layout parser, the inset
rule moved into Screen, the scissor stack moved into the base Window
(about 110 duplicated lines), named shadow and evdev constants, trimmed
includes, a README note about the Wayland dependencies, and the long
lines introduced by this branch wrapped.
Batch2D cached the bound texture and skipped the bind when the cache
matched, but the render pipeline binds textures of its own and leaves a
non-zero texture unit active. A rectangle drawn without a texture (the
hud hotbar background and slot tiles) was then modulated by whatever
texture was left bound, which made the hotbar darken for a single frame
on every window resize.

The batch now draws from texture unit 0 and compares the cached texture
with the actual binding instead of trusting the cache.
The frame the native wayland backend draws gets three switches in the
display tab: a narrow 24px bar instead of the 48px one, the shadow with
the outer resize area, and redrawing the content while the window is
resized (off keeps the previous behaviour of stretching the last frame).

The settings are applied at runtime: the bar geometry is rebuilt and the
surface is resized together with the toplevel geometry, no restart needed.
The narrow bar drops the insets and the gap of its controls, so that they
sit flush against the window edge.

The switches are hidden outside of the wayland backend, which reports its
name through Window::getBackendName().
Without the shadow and the resize border composeFrame() only forced the
alpha channel, so the buffer the driver reallocated after a size change was
presented as it is, which showed up as a black frame on every resize step.
The content is now always painted from the cache and only the shadow itself
is skipped when there are no margins.

The setting is named simple-window and means the opposite of the previous
shadow-and-border: it is off by default, so the window keeps its shadow and
the outer border for resizing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant